On the k-Means/Median Cost Function

In this work, we study the k-means cost function. The (Euclidean) k-means problem can be described as follows: given a dataset X ⊆ R and a positive integer k, find a set of k centers C ⊆ R such that Φ(C,X) def = ∑ x∈X minc∈C ||x− c|| 2 is minimized. Let ∆k(X) def = minC⊆Rd Φ(C,X) denote the cost of the optimal k-means solution. It is simple to observe that for any dataset X, ∆k(X) decreases as k increases. We try to understand this behaviour more precisely. For any dataset X ⊆ R, integer k ≥ 1, and a small precision parameter ε > 0, let L X denote the smallest integer such that ∆Lk,ε X (X) ≤ ε ·∆k(X). We show upper and lower bounds on this quantity. Our techniques generalize for the metric k-median problem in arbitrary metrics and we give bounds in terms of the doubling dimension of the metric. Finally, we observe that for any dataset X, we can compute a set S of size O ( L ε c X ) such that ∆S(X) ≤ ε ·∆k(X) using the D-sampling algorithm which is also known as the k-means++ seeding procedure. In the previous statement, c is some fixed constant. Some applications of our bounds are as follows: 1. Pseudo-approximation of k-means++: Analysing the approximation and pseudo-approximation guarantees of k-means++ seeding has been a popular research topic. The goal has been to understand how the cost behaves as a function of the number of centers sampled by this algorithm. Our results may be seen as non-trivial addition to the current state of knowledge. 2. Sampling based coreset for k-means: Our bounds imply that any constant approximation algorithm when executed with number of clusters O ( L ε2 c X ) gives an (k, ε)-coreset for the k-means problem. In particular, this means that any set S of size O ( L ε2 c X ) sampled with D-sampling is a (k, ε)-coreset. This gives an improvement over similar results of Ackermann et al. [1].